Capstone Project - The Battle of the Neighborhoods

Applied Data Science Capstone by IBM/Coursera

Introduction: Business Problem

In this project we will try to find an optimal location for a restaurant. Specifically, given the ever increasing demand (and scarcity of options) for gluten-free options in restaurants, this report will be targeted to stakeholders interested in opening a Gluten-free restaurant in Berlin, Spain.

Since there are lots of restaurants in Berlin we will try to detect locations that are not already crowded with restaurants. We are also particularly interested in areas with no Gluten-free restaurants in vicinity. We would also prefer locations as close to city center as possible, assuming that first two conditions are met.

We will use data science to generate a few most promissing neighborhoods based on this criteria. Advantages of each area will then be clearly expressed so that best possible final location can be chosen by stakeholders.

Data

Based on definition of our problem, factors that will influence our decission are:

  • number of existing restaurants in the neighborhood (any type of restaurant)
  • number of and distance to Gluten Free restaurants in the neighborhood, if any
  • distance of neighborhood from city center

I decided to use regularly spaced grid of locations, centered around city center, to define the neighborhoods.

Following data sources will be needed to extract/generate the required information:

  • centers of candidate areas will be generated algorithmically and approximate addresses of centers of those areas will be obtained using Google Maps API reverse geocoding
  • number of restaurants and their type and location in every neighborhood will be obtained using Foursquare API
  • coordinate of Berlin center will be obtained using Google Maps API geocoding of a well known Berlin location (Plaza del Sol)

Neighborhood Candidates

Let's create latitude & longitude coordinates for centroids of the candidate neighborhoods. We will create a grid of cells covering our area of interest which is aprox. 12x12 killometers centered around Berlin city center.

Let's first find the latitude & longitude of Berlin city center, using specific, well known address and Google Maps geocoding API.

In [49]:
import requests

def get_coordinates(api_key, address, verbose=False):
    try:
        url = 'https://maps.googleapis.com/maps/api/geocode/json?key={}&address={}'.format(api_key, address)
        response = requests.get(url).json()
        if verbose:
            print('Google Maps API JSON result =>', response)
        results = response['results']
        geographical_data = results[0]['geometry']['location'] # get geographical coordinates
        lat = geographical_data['lat']
        lon = geographical_data['lng']
        return [lat, lon]
    except:
        return [None, None]

google_api_key = 'AIzaSyDNcjAtExNNfM-pzsQEOnF_Z6HIFJoRLA4'
address = 'Alexanderplatz, Berlin, Germany'
madrid_center = get_coordinates(google_api_key, address)
print('Coordinate of {}: {}'.format(address, madrid_center))
Coordinate of Alexanderplatz, Berlin, Germany: [52.52198139999999, 13.413306]

Now let's create a grid of area candidates, equaly spaced, centered around city center and within ~6km from Plaza del Sol. Our neighborhoods will be defined as circular areas with a radius of 300 meters, so our neighborhood centers will be 600 meters apart.

To accurately calculate distances we need to create our grid of locations in Cartesian 2D coordinate system which allows us to calculate distances in meters (not in latitude/longitude degrees). Then we'll project those coordinates back to latitude/longitude degrees to be shown on Folium map. So let's create functions to convert between WGS84 spherical coordinate system (latitude/longitude degrees) and UTM Cartesian coordinate system (X/Y coordinates in meters).

In [50]:
#!pip install shapely
import shapely.geometry

#!pip install pyproj
import pyproj

import math

def lonlat_to_xy(lon, lat):
    proj_latlon = pyproj.Proj(proj='latlong',datum='WGS84')
    proj_xy = pyproj.Proj(proj="utm", zone=33, datum='WGS84')
    xy = pyproj.transform(proj_latlon, proj_xy, lon, lat)
    return xy[0], xy[1]

def xy_to_lonlat(x, y):
    proj_latlon = pyproj.Proj(proj='latlong',datum='WGS84')
    proj_xy = pyproj.Proj(proj="utm", zone=33, datum='WGS84')
    lonlat = pyproj.transform(proj_xy, proj_latlon, x, y)
    return lonlat[0], lonlat[1]

def calc_xy_distance(x1, y1, x2, y2):
    dx = x2 - x1
    dy = y2 - y1
    return math.sqrt(dx*dx + dy*dy)

print('Coordinate transformation check')
print('-------------------------------')
print('Madrid center longitude={}, latitude={}'.format(madrid_center[1], madrid_center[0]))
x, y = lonlat_to_xy(madrid_center[1], madrid_center[0])
print('Madrid center UTM X={}, Y={}'.format(x, y))
lo, la = xy_to_lonlat(x, y)
print('Madrid center longitude={}, latitude={}'.format(lo, la))
Coordinate transformation check
-------------------------------
Madrid center longitude=13.413306, latitude=52.52198139999999
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:18: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
Madrid center UTM X=392347.62823280576, Y=5820280.114077939
Madrid center longitude=13.413306, latitude=52.52198139999998

Let's create a hexagonal grid of cells: we offset every other row, and adjust vertical row spacing so that every cell center is equally distant from all it's neighbors.

In [ ]:
madrid_center_x, madrid_center_y = lonlat_to_xy(madrid_center[1], madrid_center[0]) # City center in Cartesian coordinates

k = math.sqrt(3) / 2 # Vertical offset for hexagonal grid cells
x_min = madrid_center_x - 6000
x_step = 600
y_min = madrid_center_y - 6000 - (int(21/k)*k*600 - 12000)/2
y_step = 600 * k 

latitudes = []
longitudes = []
distances_from_center = []
xs = []
ys = []
for i in range(0, int(21/k)):
    y = y_min + i * y_step
    x_offset = 300 if i%2==0 else 0
    for j in range(0, 21):
        x = x_min + j * x_step + x_offset
        distance_from_center = calc_xy_distance(madrid_center_x, madrid_center_y, x, y)
        if (distance_from_center <= 6001):
            lon, lat = xy_to_lonlat(x, y)
            latitudes.append(lat)
            longitudes.append(lon)
            distances_from_center.append(distance_from_center)
            xs.append(x)
            ys.append(y)

print(len(latitudes), 'candidate neighborhood centers generated.')

Let's visualize the data we have so far: city center location and candidate neighborhood centers:

In [52]:
import folium

map_madrid = folium.Map(location=madrid_center, zoom_start=13)
folium.Marker(madrid_center, popup='Alexanderplatz').add_to(map_madrid)
for lat, lon in zip(latitudes, longitudes):
    #folium.CircleMarker([lat, lon], radius=2, color='blue', fill=True, fill_color='blue', fill_opacity=1).add_to(map_berlin) 
    folium.Circle([lat, lon], radius=300, color='blue', fill=False).add_to(map_madrid)
    #folium.Marker([lat, lon]).add_to(map_madrid)
map_madrid
Out[52]:
Make this Notebook Trusted to load map: File -> Trust Notebook

OK, we now have the coordinates of centers of neighborhoods/areas to be evaluated, equally spaced (distance from every point to it's neighbors is exactly the same) and within ~6km from Plaza del Sol.

Let's now use Google Maps API to get approximate addresses of those locations.

In [53]:
def get_address(api_key, latitude, longitude, verbose=False):
    try:
        url = 'https://maps.googleapis.com/maps/api/geocode/json?key={}&latlng={},{}'.format(api_key, latitude, longitude)
        response = requests.get(url).json()
        if verbose:
            print('Google Maps API JSON result =>', response)
        results = response['results']
        address = results[0]['formatted_address']
        return address
    except:
        return None

addr = get_address(google_api_key, madrid_center[0], madrid_center[1])
print('Reverse geocoding check')
print('-----------------------')
print('Address of [{}, {}] is: {}'.format(madrid_center[0], madrid_center[1], addr))
Reverse geocoding check
-----------------------
Address of [52.52198139999999, 13.413306] is: Alexanderplatz, 10178 Berlin, Germany
In [66]:
print('Obtaining location addresses: ', end='')
addresses = []
for lat, lon in zip(latitudes, longitudes):
    address = get_address(google_api_key, lat, lon)
    if address is None:
        address = 'NO ADDRESS'
    address = address.replace(', Germany', '') # We don't need country part of address
    addresses.append(address)
    print(' .', end='')
print(' done.')
Obtaining location addresses:  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . done.
In [67]:
addresses[:10]
Out[67]:
['Ringbahnstraße 72, 12099 Berlin',
 'Unnamed Road, 12101 Berlin',
 'Unnamed Road, 12101 Berlin',
 'Hundewiese II Tempelhofer Feld, Oderstraße 174, 12049 Berlin',
 'Warthestraße 22, 12051 Berlin',
 'Altenbraker Str. 13, 12053 Berlin',
 'Karl-Marx-Straße 211, 12055 Berlin',
 'Hessenring 36, 12101 Berlin',
 'Kleineweg 129, 12101 Berlin',
 'Unnamed Road, 12101 Berlin']
In [79]:
import pandas as pd

df_locations = pd.DataFrame({'Address': addresses,
                             'Latitude': latitudes,
                             'Longitude': longitudes,
                             'X': xs,
                             'Y': ys,
                             'Distance from center': distances_from_center})

df_locations.head(10)
Out[79]:
Address Latitude Longitude X Y Distance from center
0 Ringbahnstraße 72, 12099 Berlin 52.470257 13.388666 390547.628233 5.814564e+06 5992.495307
1 Unnamed Road, 12101 Berlin 52.470377 13.397496 391147.628233 5.814564e+06 5840.376700
2 Unnamed Road, 12101 Berlin 52.470497 13.406325 391747.628233 5.814564e+06 5747.173218
3 Hundewiese II Tempelhofer Feld, Oderstraße 174... 52.470615 13.415154 392347.628233 5.814564e+06 5715.767665
4 Warthestraße 22, 12051 Berlin 52.470733 13.423984 392947.628233 5.814564e+06 5747.173218
5 Altenbraker Str. 13, 12053 Berlin 52.470851 13.432813 393547.628233 5.814564e+06 5840.376700
6 Karl-Marx-Straße 211, 12055 Berlin 52.470967 13.441643 394147.628233 5.814564e+06 5992.495307
7 Hessenring 36, 12101 Berlin 52.474746 13.375250 389647.628233 5.815084e+06 5855.766389
8 Kleineweg 129, 12101 Berlin 52.474867 13.384081 390247.628233 5.815084e+06 5604.462508
9 Unnamed Road, 12101 Berlin 52.474987 13.392911 390847.628233 5.815084e+06 5408.326913
In [80]:
df_locations.to_pickle('./locations.pkl')  

Foursquare

Now that we have our location candidates, let's use Foursquare API to get info on restaurants in each neighborhood.

We're interested in venues in 'food' category, but only those that are proper restaurants - coffe shops, pizza places, bakeries etc. are not direct competitors so we don't care about those. So we will include in out list only venues that have 'restaurant' in category name, and we'll make sure to detect and include all the subcategories of specific 'Gluten Free restaurant' category, as we need info on Gluten-free restaurants in the neighborhood.

In [82]:
CLIENT_ID = ' ' # your Foursquare ID code XDPGLDDDUILQXJZ4DQEY1AMIZMCPEAWOA3VM3SGVOGUCQ0SK#_=_
CLIENT_SECRET = ' ' # your Foursquare Secret 
ACCESS_TOKEN = ' ' # your FourSquare Access Token
VERSION = '20180604'
LIMIT = 30
In [83]:
food_category = '4d4b7105d754a06374d81259' # 'Root' category for all food-related venues

gluten_free_restaurant_category = ['4bf58dd8d48988d111941735', '55a59bace4b013909087cb30','55a59bace4b013909087cb24','55a59bace4b013909087cb2a']

def is_restaurant(categories, specific_filter=None):
    restaurant_words = ['restaurant', 'diner', 'taverna', 'steakhouse']
    restaurant = False
    specific = False
    for c in categories:
        category_name = c[0].lower()
        category_id = c[1]
        for r in restaurant_words:
            if r in category_name:
                restaurant = True
        if 'fast food' in category_name:
            restaurant = False
        if not(specific_filter is None) and (category_id in specific_filter):
            specific = True
            restaurant = True
    return restaurant, specific

def get_categories(categories):
    return [(cat['name'], cat['id']) for cat in categories]

def format_address(location):
    address = ', '.join(location['formattedAddress'])
    address = address.replace(', Deutschland', '')
    address = address.replace(', Germany', '')
    return address

def get_venues_near_location(lat, lon, category, client_id, client_secret, radius=500, limit=100):
    version = '20180724'
    url = 'https://api.foursquare.com/v2/venues/explore?client_id={}&client_secret={}&v={}&ll={},{}&categoryId={}&radius={}&limit={}'.format(
        client_id, client_secret, version, lat, lon, category, radius, limit)
    try:
        results = requests.get(url).json()['response']['groups'][0]['items']
        venues = [(item['venue']['id'],
                   item['venue']['name'],
                   get_categories(item['venue']['categories']),
                   (item['venue']['location']['lat'], item['venue']['location']['lng']),
                   format_address(item['venue']['location']),
                   item['venue']['location']['distance']) for item in results]        
    except:
        venues = []
    return venues
In [ ]:
# Let's now go over our neighborhood locations and get nearby restaurants; we'll also maintain a dictionary of all found restaurants and all found gluten-free restaurants

import pickle

def get_restaurants(lats, lons):
    restaurants = {}
    gluten_free_restaurants = {}
    location_restaurants = []

    print('Obtaining venues around candidate locations:', end='')
    for lat, lon in zip(lats, lons):
        # Using radius=350 to meke sure we have overlaps/full coverage so we don't miss any restaurant (we're using dictionaries to remove any duplicates resulting from area overlaps)
        venues = get_venues_near_location(lat, lon, food_category, CLIENT_ID, CLIENT_SECRET, radius=350, limit=100)
        area_restaurants = []
        for venue in venues:
            venue_id = venue[0]
            venue_name = venue[1]
            venue_categories = venue[2]
            venue_latlon = venue[3]
            venue_address = venue[4]
            venue_distance = venue[5]
            is_res, is_gluten_free = is_restaurant(venue_categories, specific_filter=gluten_free_restaurant_category)
            if is_res:
                x, y = lonlat_to_xy(venue_latlon[1], venue_latlon[0])
                restaurant = (venue_id, venue_name, venue_latlon[0], venue_latlon[1], venue_address, venue_distance, is_gluten_free, x, y)
                if venue_distance<=300:
                    area_restaurants.append(restaurant)
                restaurants[venue_id] = restaurant
                if is_gluten_free:
                    gluten_free_restaurants[venue_id] = restaurant
        location_restaurants.append(area_restaurants)
        print(' .', end='')
    print(' done.')
    return restaurants, gluten_free_restaurants, location_restaurants

# Try to load from local file system in case we did this before
restaurants = {}
gluten_free_restaurants = {}
location_restaurants = []
loaded = False
try:
    with open('restaurants_350.pkl', 'rb') as f:
        restaurants = pickle.load(f)
    with open('gluten_free_restaurants.pkl', 'rb') as f:
        gluten_free_restaurants = pickle.load(f)
    with open('location_restaurants_350.pkl', 'rb') as f:
        location_restaurants = pickle.load(f)
    print('Restaurant data loaded.')
    loaded = True
except:
    pass

# If load failed use the Foursquare API to get the data
if not loaded:
    restaurants, gluten_free_restaurants, location_restaurants = get_restaurants(latitudes, longitudes)
    
    # Let's persists this in local file system
    with open('restaurants_350.pkl', 'wb') as f:
        pickle.dump(restaurants, f)
    with open('gluten_free_restaurants.pkl', 'wb') as f:
        pickle.dump(gluten_free_restaurants, f)
    with open('location_restaurants_350.pkl', 'wb') as f:
        pickle.dump(location_restaurants, f)
        
In [85]:
import numpy as np

print('Total number of restaurants:', len(restaurants))
print('Total number of Gluten-free restaurants:', len(gluten_free_restaurants))
print('Percentage of Gluten-free restaurants: {:.2f}%'.format(len(gluten_free_restaurants) / len(restaurants) * 100))
print('Average number of restaurants in neighborhood:', np.array([len(r) for r in location_restaurants]).mean())
Total number of restaurants: 1558
Total number of Gluten-free restaurants: 35
Percentage of Gluten-free restaurants: 2.25%
Average number of restaurants in neighborhood: 3.6785714285714284
In [86]:
print('List of all restaurants')
print('-----------------------')
for r in list(restaurants.values())[:10]:
    print(r)
print('...')
print('Total:', len(restaurants))
List of all restaurants
-----------------------
('529c75dc498e5f28c08db637', 'Orient Food', 52.468418, 13.385631, 'Tempelhofer damm 124, 12099 Berlin', 290, False, 390336.8958508584, 5814364.380615842)
('52b5d5d0498edc50653df469', 'Yasaka-Sushi', 52.46877834428962, 13.385458971242336, 'Tempelhofer Damm 124, 12099 Berlin', 272, False, 390326.1066713873, 5814404.719417979)
('4e68e3beae60186280aeda86', 'Mahatma', 52.47215311646854, 13.38533023091156, 'Tempelhofer Damm 102 (Manfred-von-Richthofen-Str.), 12101 Berlin', 313, False, 390325.75282294105, 5814780.258426609)
('53bfbf97498e767d04eba262', 'My Asia', 52.46863425153149, 13.38513817439509, 'Tempelhofer Damm 122 (Ringahnstraße), 12103 Berlin', 299, False, 390303.9585045325, 5814389.180474727)
('4f53b76de4b0754d59050702', 'Yoko Sushi', 52.46798, 13.38516391, 'Tempelhofer Damm 128, 12099 Berlin', 347, False, 390304.07978736167, 5814316.3752975585)
('53820ddf498e1b7adfc5c1d0', 'Korean BBQ', 52.46780776977539, 13.399324417114258, 'Berlin', 311, False, 391265.5148455473, 5814275.810391908)
('50ce405fe4b0aceb60754dbc', 'Merakli Köfteci', 52.471037770421816, 13.429444253096332, 'Hermannstr 174, 12051 Berlin', 229, False, 393319.24466957763, 5814590.143150846)
('51474e67e4b02de9576b8ec6', 'STAR Gemüse-Schawarma & Falafel', 52.4703494828974, 13.429663296488343, 'Berlin', 220, False, 393332.45803737885, 5814513.267629099)
('4ea1c4f3b80355a981747dc8', 'From Hanoi With Love', 52.47136913441411, 13.429269521956977, 'Hermannstr. 176, 12051 Berlin', 247, False, 393308.17819297453, 5814627.255865358)
('4e7793c48130fa6bd3b581ff', 'Ninì e Pettirosso', 52.470317, 13.43698, 'Selkestr. 27, 12051 Berlin', 324, False, 393829.34645155125, 5814498.875631407)
...
Total: 1558
In [87]:
print('List of Gluten-free restaurants')
print('---------------------------')
for r in list(gluten_free_restaurants.values())[:10]:
    print(r)
print('...')
print('Total:', len(gluten_free_restaurants))
List of Gluten-free restaurants
---------------------------
('5709f9a5498e75ddbce8029b', 'Akai Shima No Hana', 52.484922, 13.361585, 'Leberstr. 9, 10829 Berlin', 322, True, 388745.2205669581, 5816236.73514873)
('52867b50498e334c39fce656', 'Dr. To', 52.485956, 13.434858, 'Weichselstrasse 54, 12045 Berlin', 171, True, 393722.90968280623, 5816241.385440319)
('5c1a3d1cb1538e002cb8b245', 'Qilin Kitchen', 52.490988, 13.361045, 'Hauptstr. 163, 10827 Berlin', 270, True, 388723.86754832376, 5816912.22990472)
('4e12a524e4cdef074b7e4c24', 'Cocoro', 52.4908357330855, 13.386443853378296, 'Mehringdamm 64 (Hagelberger Str.), 10961 Berlin', 235, True, 390447.8146816036, 5816856.460256244)
('59121f7bc21cb12eb99b158e', 'Life Berlin', 52.491878642334974, 13.433209737075074, 'Maybachufer 39 (Nansenstr.), 12047 Berlin', 341, True, 393625.2879796064, 5816902.53680096)
('5cba0cd39e3b650039f9bebb', 'Heo', 52.495402, 13.384372, 'Großbeerenstr., 10963 Berlin', 217, True, 390318.51946652384, 5817367.46916847)
('5d5454b840837100092fc8c5', 'Cocolo Ramen', 52.493752, 13.418159, 'Graefestr. 11 (Böckhstr.), 10967 Berlin', 49, True, 392608.07335969503, 5817133.170915079)
('57d1536f498e691ec42d6b47', 'Gobento Shiro', 52.494766, 13.432994, 'Reichenberger Str. 118, 10999 Berlin', 237, True, 393617.6108496781, 5817223.990773381)
('5c716825492822002d6493cd', 'Ohana Viethawaiian Poke', 52.4951894, 13.4324715, 'Reichenberger Str. 120, 10999 Berlin', 285, True, 393583.1634571634, 5817271.851831881)
('58a0386d9b615c0bda08da79', 'Tsukushiya - Japanese Deli', 52.50117562711077, 13.416808742410739, 'Dresdener Str. 16, 10999 Berlin', 266, True, 392534.51388605294, 5817960.845432075)
...
Total: 35
In [88]:
print('Restaurants around location')
print('---------------------------')
for i in range(100, 110):
    rs = location_restaurants[i][:8]
    names = ', '.join([r[1] for r in rs])
    print('Restaurants around location {}: {}'.format(i+1, names))
Restaurants around location
---------------------------
Restaurants around location 101: Mabuhay, Scandic Restaurant, Tischlein deck dich
Restaurants around location 102: Solar, Mirami - Sushi & Asia fusion cuisin, Layla, Ristorante Marinelli, Mexican, Cucina Italiana, Restaurant Hof zwei, Gaststätte Italia 90
Restaurants around location 103: NaNum, Nobelhart & Schmutzig, Charlotte 1, Mama Cook, Paracas, Steakhaus Asador, Café Nullpunkt, Orient Food
Restaurants around location 104: 
Restaurants around location 105: Shishi, Pacifico
Restaurants around location 106: Die Henne, Zur kleinen Markthalle, Cevichería, Parantez, Habibi, Maroush, Mawal, Santa Maria
Restaurants around location 107: La Piadina, Der Goldene Hahn, 3 Schwestern, Weltrestaurant Markthalle, Long March Canteen, Olive
Restaurants around location 108: Salumeria Lamuri, Restaurant Richard
Restaurants around location 109: Vincent Vegan, Scheers Schnitzel, Michelberger Restaurant, Tony Roma's, Seoulkitchen Korean BBQ & Sushi, Riogrande, Asia Bistro, Nano Falafel
Restaurants around location 110: Ba Qué, Areti, Kotai Asia, Opera Restaurant and Bar, Asia Food Store "We lunch", La Cesta

Let's now see all the collected restaurants in our area of interest on map, and let's also show Italian restaurants in different color.

In [89]:
map_madrid = folium.Map(location=madrid_center, zoom_start=13)
folium.Marker(madrid_center, popup='Plaza del Sol').add_to(map_madrid)
for res in restaurants.values():
    lat = res[2]; lon = res[3]
    is_italian = res[6]
    color = 'red' if is_italian else 'blue'
    folium.CircleMarker([lat, lon], radius=3, color=color, fill=True, fill_color=color, fill_opacity=1).add_to(map_madrid)
map_madrid
Out[89]:
Make this Notebook Trusted to load map: File -> Trust Notebook

Looking good. So now we have all the restaurants in area within few kilometers from Plaza del Sol, and we know which ones are Gluten-free restaurants! We also know which restaurants exactly are in vicinity of every neighborhood candidate center.

This concludes the data gathering phase - we're now ready to use this data for analysis to produce the report on optimal locations for a new Gluten-free restaurant!

Methodology

In this project we will direct our efforts on detecting areas of Madrid that have low restaurant density, particularly those with low number of Gluten-free restaurants. We will limit our analysis to area ~6km around city center.

In first step we have collected the required data: location and type (category) of every restaurant within 6km from Madrid center (Plaza del Sol). We have also identified Gluten-free restaurants (according to Foursquare categorization).

Second step in our analysis will be calculation and exploration of 'restaurant density' across different areas of Madrid - we will use heatmaps to identify a few promising areas close to center with low number of restaurants in general (and no Gluten-free restaurants in vicinity) and focus our attention on those areas.

In third and final step we will focus on most promising areas and within those create clusters of locations that meet some basic requirements established in discussion with stakeholders: we will take into consideration locations with no more than two restaurants in radius of 250 meters, and we want locations without Gluten-free restaurants in radius of 400 meters. We will present map of all such locations but also create clusters (using k-means clustering) of those locations to identify general zones / neighborhoods / addresses which should be a starting point for final 'street level' exploration and search for optimal venue location by stakeholders.

Analysis

Let's perform some basic explanatory data analysis and derive some additional info from our raw data. First let's count the number of restaurants in every area candidate:

In [90]:
location_restaurants_count = [len(res) for res in location_restaurants]

df_locations['Restaurants in area'] = location_restaurants_count
print('Average number of restaurants in every area with radius=300m:', np.array(location_restaurants_count).mean())

df_locations.head(10)
Average number of restaurants in every area with radius=300m: 3.6785714285714284
Out[90]:
Address Latitude Longitude X Y Distance from center Restaurants in area
0 Ringbahnstraße 72, 12099 Berlin 52.470257 13.388666 390547.628233 5.814564e+06 5992.495307 3
1 Unnamed Road, 12101 Berlin 52.470377 13.397496 391147.628233 5.814564e+06 5840.376700 0
2 Unnamed Road, 12101 Berlin 52.470497 13.406325 391747.628233 5.814564e+06 5747.173218 0
3 Hundewiese II Tempelhofer Feld, Oderstraße 174... 52.470615 13.415154 392347.628233 5.814564e+06 5715.767665 0
4 Warthestraße 22, 12051 Berlin 52.470733 13.423984 392947.628233 5.814564e+06 5747.173218 0
5 Altenbraker Str. 13, 12053 Berlin 52.470851 13.432813 393547.628233 5.814564e+06 5840.376700 7
6 Karl-Marx-Straße 211, 12055 Berlin 52.470967 13.441643 394147.628233 5.814564e+06 5992.495307 6
7 Hessenring 36, 12101 Berlin 52.474746 13.375250 389647.628233 5.815084e+06 5855.766389 0
8 Kleineweg 129, 12101 Berlin 52.474867 13.384081 390247.628233 5.815084e+06 5604.462508 0
9 Unnamed Road, 12101 Berlin 52.474987 13.392911 390847.628233 5.815084e+06 5408.326913 0

OK, now let's calculate the distance to nearest Gluten-free restaurant from every area candidate center (not only those within 300m - we want distance to closest one, regardless of how distant it is).

In [91]:
distances_to_gluten_free_restaurant = []

for area_x, area_y in zip(xs, ys):
    min_distance = 10000
    for res in gluten_free_restaurants.values():
        res_x = res[7]
        res_y = res[8]
        d = calc_xy_distance(area_x, area_y, res_x, res_y)
        if d<min_distance:
            min_distance = d
    distances_to_gluten_free_restaurant.append(min_distance)

df_locations['Distance to Gluten-free restaurant'] = distances_to_gluten_free_restaurant
df_locations.head()
Out[91]:
Address Latitude Longitude X Y Distance from center Restaurants in area Distance to Gluten-free restaurant
0 Ringbahnstraße 72, 12099 Berlin 52.470257 13.388666 390547.628233 5.814564e+06 5992.495307 3 2294.286080
1 Unnamed Road, 12101 Berlin 52.470377 13.397496 391147.628233 5.814564e+06 5840.376700 0 2396.565225
2 Unnamed Road, 12101 Berlin 52.470497 13.406325 391747.628233 5.814564e+06 5747.173218 0 2591.176703
3 Hundewiese II Tempelhofer Feld, Oderstraße 174... 52.470615 13.415154 392347.628233 5.814564e+06 5715.767665 0 2168.838160
4 Warthestraße 22, 12051 Berlin 52.470733 13.423984 392947.628233 5.814564e+06 5747.173218 0 1847.571711
In [92]:
print('Average distance to closest Gluten-free restaurant from each area center:', df_locations['Distance to Gluten-free restaurant'].mean())
Average distance to closest Gluten-free restaurant from each area center: 1759.0565614546242

OK, so on average Gluten-free restaurant can be found within ~500m from every area center candidate. That's fairly close, so we need to filter our areas carefully! Let's crete a map showing heatmap / density of restaurants and try to extract some meaningfull info from that. Also, let's show borders of Berlin boroughs on our map and a few circles indicating distance of 1km, 2km and 3km from Plaza del Sol.

In [93]:
berlin_boroughs_url = 'https://raw.githubusercontent.com/m-hoerz/berlin-shapes/master/berliner-bezirke.geojson'
berlin_boroughs = requests.get(berlin_boroughs_url).json()

def boroughs_style(feature):
    return { 'color': 'blue', 'fill': False }
In [94]:
restaurant_latlons = [[res[2], res[3]] for res in restaurants.values()]
gluten_free_latlons = [[res[2], res[3]] for res in gluten_free_restaurants.values()]
from folium import plugins
from folium.plugins import HeatMap

berlin_center = madrid_center
map_berlin = folium.Map(location=berlin_center, zoom_start=13)
folium.TileLayer('cartodbpositron').add_to(map_berlin) #cartodbpositron cartodbdark_matter
HeatMap(restaurant_latlons).add_to(map_berlin)
folium.Marker(berlin_center).add_to(map_berlin)
folium.Circle(berlin_center, radius=1000, fill=False, color='white').add_to(map_berlin)
folium.Circle(berlin_center, radius=2000, fill=False, color='white').add_to(map_berlin)
folium.Circle(berlin_center, radius=3000, fill=False, color='white').add_to(map_berlin)
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin
Out[94]:
Make this Notebook Trusted to load map: File -> Trust Notebook
In [96]:
map_berlin = folium.Map(location=berlin_center, zoom_start=13)
folium.TileLayer('cartodbpositron').add_to(map_berlin) #cartodbpositron cartodbdark_matter
HeatMap(gluten_free_latlons).add_to(map_berlin)
folium.Marker(berlin_center).add_to(map_berlin)
folium.Circle(berlin_center, radius=1000, fill=False, color='white').add_to(map_berlin)
folium.Circle(berlin_center, radius=2000, fill=False, color='white').add_to(map_berlin)
folium.Circle(berlin_center, radius=3000, fill=False, color='white').add_to(map_berlin)
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin
Out[96]:
Make this Notebook Trusted to load map: File -> Trust Notebook

This map is not so 'hot' but it also indicates higher density of existing Gluten-free restaurants directly north and west from Alexanderplatz, with closest pockets of low gluten-free restaurant density positioned east, south-east and south from city center.

Based on this we will now focus our analysis on areas south-west, south, south-east and east from Berlin center - we will move the center of our area of interest and reduce it's size to have a radius of 2.5km. This places our location candidates mostly in boroughs Kreuzberg and Friedrichshain (another potentially interesting borough is Prenzlauer Berg with large low restaurant density north-east from city center, however this borough is less interesting to stakeholders as it's mostly residental and less popular with tourists).

Kreuzberg and Friedrichshain

Analysis of popular travel guides and web sites often mention Kreuzberg and Friedrichshain as beautifull, interesting, rich with culture, 'hip' and 'cool' Berlin neighborhoods popular with tourists and loved by Berliners.

"Bold and brazen, Kreuzberg's creative people, places, and spaces might challenge your paradigm." Tags: Nightlife, Artsy, Dining, Trendy, Loved by Berliners, Great Transit (airbnb.com)

"Kreuzberg has long been revered for its diverse cultural life and as a part of Berlin where alternative lifestyles have flourished. Envisioning the glamorous yet gritty nature of Berlin often conjures up scenes from this neighbourhood, where cultures, movements and artistic flare adorn the walls of building and fills the air. Brimming with nightclubs, street food, and art galleries, Kreuzberg is the place to be for Berlin’s young and trendy." (theculturetrip.com)

"Imagine an art gallery turned inside out and you’ll begin to envision Friedrichshain. Single walls aren’t canvases for creative works, entire buildings are canvases. This zealously expressive east Berlin neighborhood forgoes social norms" Tags: Artsy, Nightlife, Trendy, Dining, Touristy, Shopping, Great Transit, Loved by Berliners (airbnb.com)

"As anyone from Kreuzberg will tell you, this district is not just the coolest in Berlin, but the hippest location in the entire universe. Kreuzberg has long been famed for its diverse cultural life, its experimental alternative lifestyles and the powerful spell it exercises on young people from across Germany. In 2001, Kreuzberg and Friedrichshain were merged to form one administrative borough. When it comes to club culture, Friedrichshain is now out in front – with southern Friedrichshain particularly ranked as home to the highest density of clubs in the city." (visitberlin.de)

Popular with tourists, alternative and bohemian but booming and trendy, relatively close to city center and well connected, those boroughs appear to justify further analysis.

Let's define new, more narrow region of interest, which will include low-restaurant-count parts of Kreuzberg and Friedrichshain closest to Alexanderplatz.

In [99]:
roi_x_min = madrid_center_x - 2000
roi_y_max = madrid_center_y + 1000
roi_width = 5000
roi_height = 5000
roi_center_x = roi_x_min + 2500
roi_center_y = roi_y_max - 2500
roi_center_lon, roi_center_lat = xy_to_lonlat(roi_center_x, roi_center_y)
roi_center = [roi_center_lat, roi_center_lon]

map_berlin = folium.Map(location=roi_center, zoom_start=14)
HeatMap(restaurant_latlons).add_to(map_berlin)
folium.Marker(berlin_center).add_to(map_berlin)
folium.Circle(roi_center, radius=2500, color='white', fill=True, fill_opacity=0.4).add_to(map_berlin)
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:18: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
Out[99]:
Make this Notebook Trusted to load map: File -> Trust Notebook
In [ ]:
k = math.sqrt(3) / 2 # Vertical offset for hexagonal grid cells
x_step = 100
y_step = 100 * k 
roi_y_min = roi_center_y - 2500

roi_latitudes = []
roi_longitudes = []
roi_xs = []
roi_ys = []
for i in range(0, int(51/k)):
    y = roi_y_min + i * y_step
    x_offset = 50 if i%2==0 else 0
    for j in range(0, 51):
        x = roi_x_min + j * x_step + x_offset
        d = calc_xy_distance(roi_center_x, roi_center_y, x, y)
        if (d <= 2501):
            lon, lat = xy_to_lonlat(x, y)
            roi_latitudes.append(lat)
            roi_longitudes.append(lon)
            roi_xs.append(x)
            roi_ys.append(y)

print(len(roi_latitudes), 'candidate neighborhood centers generated.')

OK. Now let's calculate two most important things for each location candidate: number of restaurants in vicinity (we'll use radius of 250 meters) and distance to closest Gluten-free restaurant.

In [103]:
def count_restaurants_nearby(x, y, restaurants, radius=250):    
    count = 0
    for res in restaurants.values():
        res_x = res[7]; res_y = res[8]
        d = calc_xy_distance(x, y, res_x, res_y)
        if d<=radius:
            count += 1
    return count

def find_nearest_restaurant(x, y, restaurants):
    d_min = 100000
    for res in restaurants.values():
        res_x = res[7]; res_y = res[8]
        d = calc_xy_distance(x, y, res_x, res_y)
        if d<=d_min:
            d_min = d
    return d_min

roi_restaurant_counts = []
roi_gluten_free_distances = []

print('Generating data on location candidates... ', end='')
for x, y in zip(roi_xs, roi_ys):
    count = count_restaurants_nearby(x, y, restaurants, radius=250)
    roi_restaurant_counts.append(count)
    distance = find_nearest_restaurant(x, y, gluten_free_restaurants)
    roi_gluten_free_distances.append(distance)
print('done.')
Generating data on location candidates... done.
In [104]:
# Let's put this into dataframe
df_roi_locations = pd.DataFrame({'Latitude':roi_latitudes,
                                 'Longitude':roi_longitudes,
                                 'X':roi_xs,
                                 'Y':roi_ys,
                                 'Restaurants nearby':roi_restaurant_counts,
                                 'Distance to Italian restaurant':roi_gluten_free_distances})

df_roi_locations.head(10)
Out[104]:
Latitude Longitude X Y Restaurants nearby Distance to Italian restaurant
0 52.486123 13.421225 392797.628233 5.816280e+06 6 873.863272
1 52.486143 13.422697 392897.628233 5.816280e+06 7 826.189675
2 52.486793 13.413100 392247.628233 5.816367e+06 0 846.978677
3 52.486813 13.414572 392347.628233 5.816367e+06 0 809.496049
4 52.486832 13.416044 392447.628233 5.816367e+06 0 783.067575
5 52.486852 13.417516 392547.628233 5.816367e+06 2 768.834054
6 52.486872 13.418988 392647.628233 5.816367e+06 5 767.474284
7 52.486891 13.420461 392747.628233 5.816367e+06 5 779.055679
8 52.486911 13.421933 392847.628233 5.816367e+06 7 803.018509
9 52.486931 13.423405 392947.628233 5.816367e+06 11 785.346567

OK. Let us now filter those locations: we're interested only in locations with no more than two restaurants in radius of 250 meters, and no Gluten-free restaurants in radius of 400 meters.

In [105]:
good_res_count = np.array((df_roi_locations['Restaurants nearby']<=2))
print('Locations with no more than two restaurants nearby:', good_res_count.sum())

good_ita_distance = np.array(df_roi_locations['Distance to Italian restaurant']>=400)
print('Locations with no Italian restaurants within 400m:', good_ita_distance.sum())


good_locations = np.logical_and(good_res_count, good_ita_distance)
print('Locations with both conditions met:', good_locations.sum())

df_good_locations = df_roi_locations[good_locations]
good_latitudes = df_good_locations['Latitude'].values
good_longitudes = df_good_locations['Longitude'].values

good_locations = [[lat, lon] for lat, lon in zip(good_latitudes, good_longitudes)]

map_berlin = folium.Map(location=roi_center, zoom_start=14)
folium.TileLayer('cartodbpositron').add_to(map_berlin)
HeatMap(restaurant_latlons).add_to(map_berlin)
folium.Circle(roi_center, radius=2500, color='white', fill=True, fill_opacity=0.6).add_to(map_berlin)
folium.Marker(berlin_center).add_to(map_berlin)
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.CircleMarker([lat, lon], radius=2, color='blue', fill=True, fill_color='blue', fill_opacity=1).add_to(map_berlin) 
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin
Locations with no more than two restaurants nearby: 809
Locations with no Italian restaurants within 400m: 1715
Locations with both conditions met: 763
Out[105]:
Make this Notebook Trusted to load map: File -> Trust Notebook

Looking good. We now have a bunch of locations fairly close to Alexanderplatz (mostly in Kreuzberg, Friedrichshain and south-east corner of Mitte boroughs), and we know that each of those locations has no more than two restaurants in radius of 250m, and no Gluten-free restaurant closer than 400m. Any of those locations is a potential candidate for a new Gluten-free restaurant, at least based on nearby competition.

Let's now show those good locations in a form of heatmap:

In [106]:
map_berlin = folium.Map(location=roi_center, zoom_start=14)
HeatMap(good_locations, radius=25).add_to(map_berlin)
folium.Marker(berlin_center).add_to(map_berlin)
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.CircleMarker([lat, lon], radius=2, color='blue', fill=True, fill_color='blue', fill_opacity=1).add_to(map_berlin)
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin
Out[106]:
Make this Notebook Trusted to load map: File -> Trust Notebook

Looking good. What we have now is a clear indication of zones with low number of restaurants in vicinity, and no Gluten-free restaurants at all nearby.

Let us now cluster those locations to create centers of zones containing good locations. Those zones, their centers and addresses will be the final result of our analysis.

In [ ]:
from sklearn.cluster import KMeans

number_of_clusters = 15

good_xys = df_good_locations[['X', 'Y']].values
kmeans = KMeans(n_clusters=number_of_clusters, random_state=0).fit(good_xys)

cluster_centers = [xy_to_lonlat(cc[0], cc[1]) for cc in kmeans.cluster_centers_]

map_berlin = folium.Map(location=roi_center, zoom_start=14)
folium.TileLayer('cartodbpositron').add_to(map_berlin)
HeatMap(restaurant_latlons).add_to(map_berlin)
folium.Circle(roi_center, radius=2500, color='white', fill=True, fill_opacity=0.4).add_to(map_berlin)
folium.Marker(berlin_center).add_to(map_berlin)
for lon, lat in cluster_centers:
    folium.Circle([lat, lon], radius=500, color='green', fill=True, fill_opacity=0.25).add_to(map_berlin) 
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.CircleMarker([lat, lon], radius=2, color='blue', fill=True, fill_color='blue', fill_opacity=1).add_to(map_berlin)
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin

Not bad - our clusters represent groupings of most of the candidate locations and cluster centers are placed nicely in the middle of the zones 'rich' with location candidates.

Addresses of those cluster centers will be a good starting point for exploring the neighborhoods to find the best possible location based on neighborhood specifics.

Let's see those zones on a city map without heatmap, using shaded areas to indicate our clusters:

In [108]:
map_berlin = folium.Map(location=roi_center, zoom_start=14)
folium.Marker(berlin_center).add_to(map_berlin)
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.Circle([lat, lon], radius=250, color='#00000000', fill=True, fill_color='#0066ff', fill_opacity=0.07).add_to(map_berlin)
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.CircleMarker([lat, lon], radius=2, color='blue', fill=True, fill_color='blue', fill_opacity=1).add_to(map_berlin)
for lon, lat in cluster_centers:
    folium.Circle([lat, lon], radius=500, color='green', fill=False).add_to(map_berlin) 
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin
Out[108]:
Make this Notebook Trusted to load map: File -> Trust Notebook

Let's zoom in on candidate areas in Kreuzberg:

In [109]:
map_berlin = folium.Map(location=[52.498972, 13.409591], zoom_start=15)
folium.Marker(berlin_center).add_to(map_berlin)
for lon, lat in cluster_centers:
    folium.Circle([lat, lon], radius=500, color='green', fill=False).add_to(map_berlin) 
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.Circle([lat, lon], radius=250, color='#0000ff00', fill=True, fill_color='#0066ff', fill_opacity=0.07).add_to(map_berlin)
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.CircleMarker([lat, lon], radius=2, color='blue', fill=True, fill_color='blue', fill_opacity=1).add_to(map_berlin)
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin
Out[109]:
Make this Notebook Trusted to load map: File -> Trust Notebook

...and candidate areas in Friedrichshain:

In [110]:
map_berlin = folium.Map(location=[52.516347, 13.428403], zoom_start=15)
folium.Marker(berlin_center).add_to(map_berlin)
for lon, lat in cluster_centers:
    folium.Circle([lat, lon], radius=500, color='green', fill=False).add_to(map_berlin) 
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.Circle([lat, lon], radius=250, color='#0000ff00', fill=True, fill_color='#0066ff', fill_opacity=0.07).add_to(map_berlin)
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.CircleMarker([lat, lon], radius=2, color='blue', fill=True, fill_color='blue', fill_opacity=1).add_to(map_berlin)
folium.GeoJson(berlin_boroughs, style_function=boroughs_style, name='geojson').add_to(map_berlin)
map_berlin
Out[110]:
Make this Notebook Trusted to load map: File -> Trust Notebook

Finaly, let's reverse geocode those candidate area centers to get the addresses which can be presented to stakeholders.

In [112]:
candidate_area_addresses = []
print('==============================================================')
print('Addresses of centers of areas recommended for further analysis')
print('==============================================================\n')
for lon, lat in cluster_centers:
    addr = get_address(google_api_key, lat, lon).replace(', Germany', '')
    candidate_area_addresses.append(addr)    
    x, y = lonlat_to_xy(lon, lat)
    d = calc_xy_distance(x, y, madrid_center_x, madrid_center_y)
    print('{}{} => {:.1f}km from Alexanderplatz'.format(addr, ' '*(50-len(addr)), d/1000))
==============================================================
Addresses of centers of areas recommended for further analysis
==============================================================

/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Hotel Berolina, Karl-Marx-Allee 31, 10178 Berlin   => 0.7km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Alexandrinenstraße 42, 10969 Berlin                => 1.9km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Str. der Pariser Kommune 26, 10243 Berlin          => 2.0km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Lehmbruckstraße 23, 10245 Berlin                   => 3.5km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Unnamed Road, 10249 Berlin                         => 1.5km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Hasenheide 81, 10967 Berlin                        => 3.8km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Melchiorstraße 34, 10179 Berlin                    => 1.8km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Markgrafenstraße 67, 10969 Berlin                  => 2.2km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Unnamed Road, 10249 Berlin                         => 2.0km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Carl-Herz-Ufer 22, 10961 Berlin                    => 2.9km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Prenzlauer Berg 15/16, 10405 Berlin                => 0.9km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Breite Str. 36, 10178 Berlin                       => 1.0km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
An der Ostbahn 2d-e, 10243 Berlin                  => 2.4km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':
Lohmühlenstraße 17, 12435 Berlin                   => 3.8km from Alexanderplatz
Krautstraße 27, 10243 Berlin                       => 1.3km from Alexanderplatz
/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:12: DeprecationWarning: This function is deprecated. See: https://pyproj4.github.io/pyproj/stable/gotchas.html#upgrading-to-pyproj-2-from-pyproj-1
  if sys.path[0] == '':

This concludes our analysis. We have created 15 addresses representing centers of zones containing locations with low number of restaurants and no Italian restaurants nearby, all zones being fairly close to city center (all less than 4km from Alexanderplazt, and about half of those less than 2km from Alexanderplatz). Although zones are shown on map with a radius of ~500 meters (green circles), their shape is actually very irregular and their centers/addresses should be considered only as a starting point for exploring area neighborhoods in search for potential restaurant locations. Most of the zones are located in Kreuzberg and Friedrichshain boroughs, which we have identified as interesting due to being popular with tourists, fairly close to city center and well connected by public transport.

In [113]:
map_berlin = folium.Map(location=roi_center, zoom_start=14)
folium.Circle(berlin_center, radius=50, color='red', fill=True, fill_color='red', fill_opacity=1).add_to(map_berlin)
for lonlat, addr in zip(cluster_centers, candidate_area_addresses):
    folium.Marker([lonlat[1], lonlat[0]], popup=addr).add_to(map_berlin) 
for lat, lon in zip(good_latitudes, good_longitudes):
    folium.Circle([lat, lon], radius=250, color='#0000ff00', fill=True, fill_color='#0066ff', fill_opacity=0.05).add_to(map_berlin)
map_berlin
Out[113]:
Make this Notebook Trusted to load map: File -> Trust Notebook

Results and Discussion

Our analysis shows that although there is a great number of restaurants in Berlin (~2000 in our initial area of interest which was 12x12km around Alexanderplatz), there are pockets of low restaurant density fairly close to city center. Highest concentration of restaurants was detected north and west from Alexanderplatz, so we focused our attention to areas south, south-east and east, corresponding to boroughs Kreuzberg, Friedrichshain and south-east corner of central Mitte borough. Another borough was identified as potentially interesting (Prenzlauer Berg, north-east from Alexanderplatz), but our attention was focused on Kreuzberg and Friedrichshain which offer a combination of popularity among tourists, closeness to city center, strong socio-economic dynamics and a number of pockets of low restaurant density.

After directing our attention to this more narrow area of interest (covering approx. 5x5km south-east from Alexanderplatz) we first created a dense grid of location candidates (spaced 100m appart); those locations were then filtered so that those with more than two restaurants in radius of 250m and those with an Italian restaurant closer than 400m were removed.

Those location candidates were then clustered to create zones of interest which contain greatest number of location candidates. Addresses of centers of those zones were also generated using reverse geocoding to be used as markers/starting points for more detailed local analysis based on other factors.

Result of all this is 15 zones containing largest number of potential new restaurant locations based on number of and distance to existing venues - both restaurants in general and Italian restaurants particularly. This, of course, does not imply that those zones are actually optimal locations for a new restaurant! Purpose of this analysis was to only provide info on areas close to Berlin center but not crowded with existing restaurants (particularly Italian) - it is entirely possible that there is a very good reason for small number of restaurants in any of those areas, reasons which would make them unsuitable for a new restaurant regardless of lack of competition in the area. Recommended zones should therefore be considered only as a starting point for more detailed analysis which could eventually result in location which has not only no nearby competition but also other factors taken into account and all other relevant conditions met.

Conclusion

Purpose of this project was to identify Berlin areas close to center with low number of restaurants (particularly Italian restaurants) in order to aid stakeholders in narrowing down the search for optimal location for a new Italian restaurant. By calculating restaurant density distribution from Foursquare data we have first identified general boroughs that justify further analysis (Kreuzberg and Friedrichshain), and then generated extensive collection of locations which satisfy some basic requirements regarding existing nearby restaurants. Clustering of those locations was then performed in order to create major zones of interest (containing greatest number of potential locations) and addresses of those zone centers were created to be used as starting points for final exploration by stakeholders.

Final decission on optimal restaurant location will be made by stakeholders based on specific characteristics of neighborhoods and locations in every recommended zone, taking into consideration additional factors like attractiveness of each location (proximity to park or water), levels of noise / proximity to major roads, real estate availability, prices, social and economic dynamics of every neighborhood etc.